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FOREWORD 


The  Navy  is  planning  to  develop  an  instrument  to  measure  vocational  interests  that  will  be 
ultimately  administered  in  conjunction  with  the  Armed  Services  Vocational  Aptitude  Battery  (AS- 
VAB).  This  report  reviews  the  literature  concerning  vocational  interest  assessment,  and  measures 
high  school  counselor  preferences  on  the  format  and  results  reporting  methods  that  the  interest  in¬ 
strument  should  exhibit. 

This  research  was  funded  through  the  Office  of  the  Assistant  Secretary  of  Defense  (FM&P). 
The  results  of  this  report  are  expected  to  benefit  Navy  recruiting  as  well  as  the  research  community. 

RICHARD  C.  SORENSON 
Director,  Personnel  Systems  Department 
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Problem 


In  order  to  increase  the  number  of  qualified  personnel  entering  the  military,  the  Navy  Person¬ 
nel  Research  and  Development  Center  is  designing  and  developing  a  vocational  interest  assess¬ 
ment  instrument,  the  Armed  Services  Vocational  Interest  Profile  (ASVIP),  that  is  compatible  with 
the  high  school  testing  and  counseling  environment  and  could  also  be  used  in  conjunction  with  the 
Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  results. 


Objectives 

The  objectives  of  this  study  were  to:  ( 1 )  assess  counselor  preferences  on  the  format  and  results 
reporting  methods  the  interest  instrument  should  exhibit,  and  (2)  assess  the  acceptance  and  use  of 
a  new  interest  assessment  instrument  by  high  school  counselors.  /"  ^ 

Method 


Subjects  for  the  study  were  high  school  guidance  counselors  from  a  local  school  district.  A 
total  of  12  counselors  participated;  eight  women  and  four  men  (with  one  black  male  and  one  black 
female).  The  counselors  were  divided  into  two  discussion  groups  with  six  counselors  in  each 
group.  A  small  group,  structured  interviewing  approach  (i.e.,  moderated  discussion  groups)  was 
used  as  the  means  for  data  collection.  During  the  discussion  groups,  other  issues  that  were  of  im¬ 
portance  to  counselor  acceptance  of  a  new  interest  assessment  instrument  were  also  addressed. 

Results  and  Conclusions 

The  design  features  this  group  of  counselors  would  prefer  in  a  new  interest  assessment  instru¬ 
ment  in  rank  order  are: 


1.  Results  linked  with  a  measure  of  aptitude  and  sources  of  occupational  information. 

2.  Inexpensive  and  effective. 

3.  Unbiased  results. 

4.  Self  scoring  and  interpretable. 

5.  Administration  time  should  be  about  30  minutes. 

6.  Results  should  be  in  terms  of  occupational  themes  vs.  occupational  scales. 

7.  Items  should  be  in  terms  of  activities  associated  with  occupations  vs.  occupational 
titles. 


8.  All  documentation,  items,  and  results  should  be  written  at  an  appro,  iriate  reading  level. 

9.  Results  should  be  reported  in  terms  of  combined-sex  scores. 

10.  Instrument  should  be  attractively  formatted  and  easy  to  read. 
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INTRODUCTION 


Problem 

In  order  to  increase  the  number  of  qualified  personnel  entering  the  military,  the  Navy 
Personnel  Research  and  Development  Center  is  designing  and  developing  a  vocational  interest 
assessment  instrument,  the  Armed  Services  Vocational  Interest  Profile  (AS VIP),  that  is  compatible 
with  the  high  school  testing  and  counseling  environment  and  could  also  be  used  in  conjunction 
with  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  results. 

Objectives 

The  objectives  of  this  study  were  to:  (1)  assess  counselor  preferences  on  the  format  and  results 
reporting  methods  the  interest  instrument  should  exhibit,  and  (2)  assess  the  acceptance  and  use  of 
a  new  interest  assessment  instrument  by  high  school  counselors. 

In  addition,  a  brief  literature  review  is  presented  on  the  different  methods  of  assessing 
vocational  interests,  use  of  interest  assessment  data,  methods  for  scoring  and  reporting  interest 
results,  and  validity  of  interest  results  in  predicting  job  performance  and  job  satisfaction. 

Background 

Various  types  of  instruments  are  used  for  assessment  purposes  in  career  counseling  and 
guidance  programs.  The  purpose  of  these  instruments  is  to  help  people  understand  themselves  in 
terms  of  their  talents,  interests,  values,  and  personality  characteristics.  Two  of  the  most  prevalent 
types  of  instruments  used  today  are  aptitude  batteries  and  interest  inventories. 

The  military  has  been  administering  the  ASVAB  to  students  for  about  20  years.  The  primary 
purposes  of  the  ASVAB  is  to  provide  the  students  with  aptitude  scores  to  assist  them  in  career 
exploration  and  decision  making.  Today,  ASVAB- 14  is  administered  by  Department  of  Defense 
(DoD)  professionals,  free  of  charge,  to  over  1  million  students  each  year  across  the  nation. 

Review  of  the  Vocational  Interest  Assessment  Literature 

The  following  is  a  brief  literature  review  of  interest  assessment  and  is  limited  to  the  following 
topics:  (1)  types  of  interest  assessment,  (2)  use  of  interest  assessment  results,  (3)  methods  of 
scoring  and  reporting  interest  assessment  results,  and  (4)  the  validity  of  interest  assessment 
instruments  in  predicting  job  performance  and  job  satisfaction. 

The  literature  review  consisted  of  a  computerized  search  using  the  PsychLit  data  base, 
obtaining  copies  of  commonly  used  interest  inventories,  and  conversations  with  prominent 
psychologists  in  career  guidance  such  as  Dr.  Edwin  Herr,  Dr.  Stanley  Cramer,  Dr.  Esther  Diamond, 
and  Dr.  Peter  Cairo  on  sources  of  information  for  the  literature  review. 

Types  of  Interest  Assessment 

Super  and  Crites  (1962)  have  observed  that  individual  interests  may  be  assessed  in  four  ways: 
(1)  manifest  interests- what  a  person  actually  does  as  an  indication  of  his  or  her  interests,  (2) 
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testing-assessing  interests  by  using  an  instrument  such  as  the  Michigan  Vocabulary  Test  on  the 
grounds  that  if  an  individual  is  really  interested  in  an  occupation,  he  or  she  will  know  the 
vocabulary  of  that  area,  (3)  expressed  interests-areas  in  which  an  individual  expresses  interest,  and 
(4)  measured  interests— determining  the  pattern  of  an  individual’s  interests  from  his  or  her 
responses  to  lists  of  occupations  or  activities  (e.g.,  interest  inventories). 

By  far,  the  most  common  method  in  use  today  for  assessing  interests  is  interest  inventories 
(Herr  &  Cramer,  1988).  One  ongoing  controversy  regarding  interest  assessment  is  the  relative 
superiority  of  measured  vs.  expressed  interests.  To  date,  this  controversy  has  not  been  resolved. 
Twenty  years  ago,  review  articles  by  Dolliver  (1969),  Holland  and  Lutz  (1968),  and  Whitney 
(1969)  reviewed  research  up  to  that  time  comparing  the  merits  of  expressed  and  inventoried 
interests.  The  authors  concluded  that  expressed  interests  had  predictive  validity  equal  to  or 
exceeding  that  of  inventoried  interests  for  selecting  a  college  major  or  career  (Slaney  &  Slaney, 
1986).  More  recent  reviews  by  Baker  (1983),  Borgen  (1986),  and  Slaney  and  Slaney  (1986)  have 
found  additional  support  for  the  predictive  validity  of  expressed  vocational  choice  (e.g.,  Apostal, 
1985;  Athanasou  &  Evans,  1983;  Bartling  &  Hood,  1981;  Borgen  &  Seling,  1978;  Cairo,  1982; 
Holland  &  Gottfredson,  1975;  Laing,  Swaney,  &  Prediger,  1984;  Slaney,  1978;  and  Slaney,  1984). 

According  to  Borgen  (1986),  several  of  the  above  studies  were  inspired  by  the  question  of 
what  to  trust— person  or  test— when  they  disagree.  Borgen  states  that  thus  far,  research  results  show 
that  when  the  two  disagree,  expressed  choices  often  had  superior  predictive  or  concurrent  validity 
to  measured  interests.  When  both  agreed,  predictive  validity  was  even  better.  Laing  et  al.  (1984) 
found  that  persistence  in  an  expressed  choice  increases  systematically  as  congruence  between 
measured  interests  and  that  choice  increases. 

Although  there  has  been  a  long  history  of  research  supporting  the  use  of  expressed  interests 
in  career  guidance,  career  counselors  have  been  extremely  slow  to  increase  the  use  of  expressed 
interests  in  counseling  (Borgen  &  Seling,  1978).  According  to  Hansen  (1984),  counselors  may  be 
reluctant  to  use  expressed  interests  as  a  primary  means  for  interest  assessment  because  they 
believe:  (1)  estimates  of  interest  at  a  young  age  probably  lack  a  sense  of  reality;  (2)  the  large 
number  of  items  in  an  interest  inventory  provide  a  more  thorough  sampling  of  interests;  (3) 
expressed  interests  are  less  permanent  than  inventoried  interests;  and  (4)  expressed  interests  are 
unduly  influenced  by  factors  such  as  prestige,  family  pressure,  and  misconceptions  of  the  world  of 
work. 

Hansen  (1984)  is  also  skeptical  of  the  validity  of  expressed  interests  due  to  what  she  feels  are 
inadequacies  in  the  methodological  designs  in  most  of  the  studies  investigating  the  predictive 
validity  of  expressed  interests. 

According  to  Hansen  (1984),  the  preferred  mode  of  vocational  exploration  now  is  the 
integration  of  expressed  and  inventoried  interests.  For  students  who  have  a  definite  occupational 
choice,  interest  inventory  scores  may  serve  as  confirmation  of  choices  already  expressed  during 
counseling.  Conflicting  results  between  expressed  and  inventoried  interests  can  lead  to  useful 
discussions  about  the  causes  of  the  discrepancies  and  to  a  better  understanding  of  an  individual’s 
motivation  for  selecting  an  occupation,  which  may  be  related  to  factors  other  than  interests. 
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Use  of  Interest  Assessment  Results 


In  one  of  the  few  studies  to  survey  the  use  of  commercially  published  tests  in  high  schools, 
Engen,  Lamb,  and  Prediger  (1982),  found  that  about  80  percent  of  all  high  schools  surveyed  used 
interest  inventories.  The  most  frequently  used  interest  inventories,  according  to  their  use  at  any 
grade  level  for  at  least  one  percent  of  the  schools  in  the  sample,  are:  (1)  Kuder  Occupational 
Interest  Survey  (Form  DD),  (2)  Strong-Campbell  Interest  Inventory  (SCII),  (3)  Kuder  Vocational 
or  General  Interest  Survey,  (4)  Ohio  Vocational  Interest  Survey  (OVIS),  (5)  Self-Directed  Search 
(SDS),  (6)  California  Occupational  Preference  Survey  (COPS)  (7)  Judgment  of  Occupational 
Behavior-Orientation  (JOB-O),  and  (8)  Vocational  Interest,  Experience  and  Skills  Assessment 
(VESA). 

Engen  et  al.  (1982)  reported  that  according  to  the  survey  results,  the  use  of  commercial  tests 
still  plays  an  integral  role  in  high  school  counseling  today.  Although  the  overall  frequency  of  test 
use  has  not  changed  greatly  in  the  last  25  years,  use  of  interest  inventories  has  increased 
appreciably  (Zytowski  &  Warman,  1982).  Also,  counselors  have  shifted  from  using  measures  of 
ability  and  achievement  to  using  interest,  value,  and  personality  inventories  (Hansen,  1984). 
Schools  indicate  they  would  do  even  more  testing  for  career  guidance  purposes  if  the  funds  were 
available  (Engen  et  al.,  1982). 

According  to  Hansen  (1981a),  interest  inventories  are  used  primarily  by  educational 
institutions,  but  the  increased  recognition  of  the  importance  of  interests  in  career  and  life-style 
satisfaction  has  led  to  the  use  of  interest  measurement  in  government,  business,  and  industry. 
Hansen  reports  that  interest  inventories  are  also  being  used  occasionally  in  personnel  areas  such  as 
selection  and  placement. 

Hansen  (1981a)  goes  on  to  explain  that  interest  inventories  are  also  useful  in  research  projects 
designed  to  explore  hypotheses  generated  from  clinical  experience.  The  hypotheses  are  tested 
against  data  and,  if  proven  true,  the  results  can  be  used  to  increase  the  counselor’s  knowledge  of 
interests  and  vocational  psychology  in  general. 

Some  descriptions  of  how  Hansen  believes  that  interest  inventories  can  be  used  by 
counselors,  clients,  personnel  specialists,  and  researchers  are  listed  below. 

Counselor  Applications.  This  includes:  (1)  generating  hypotheses  and  interviewing— one  of 
the  primary  uses  of  interest  inventory  results  for  counselors  should  be  to  generate  hypotheses  about 
the  client,  which  can  be  discussed,  explored,  and  confirmed  or  discarded  by  the  client;  (2)  selecting 
counseling  techniques-while  serving  as  an  aid  to  interviewing,  interest  inventory  scores  can  often 
give  counselors  clues  for  discovering  the  best  counseling  style  to  use  with  clients;  and  (3) 
counseling  the  handicapped-interest  inventories  can  help  the  handicapped  to  identify  interests  as 
a  starting  point  to  promote  emotional  adjustment  and  educational  and/or  occupational  satisfaction. 

Client  Applications.  This  includes:  (1)  beginning  career  exploration-scores  on  an  interest 
inventory  are  measures  of  an  individual’s  interest  in  various  activity  areas  or  occupations  and 
suggest  where  an  individual  is  most  likely  to  persist  and/or  be  happy;  (2)  making  educational 
decisions-interest  inventory  results  can  be  used  not  only  to  help  an  individual  make  a  career  choice 
but  as  guidance  for  selecting  major  courses  and  electives;  (3)  stimulating  new  interest- 
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occasionally,  a  client  who  appears  to  be  seeking  only  vocational  confirmation  through  test  data  will 
discover  other  potential  choices  that  have  not  occurred  to  her  or  him;  (4)  considering  alternative 
careers  and  life-stvles-interpretation  of  the  inventory  is  an  opportunity  for  the  counselor  to  discuss 
with  the  client  his  or  her  entire  life-style  rather  than  focus  on  vocational  interests  and  occupational 
decisions;  (5)  planning  midcareer  changes-as  a  person’s  experiences  broaden,  earlier  vocational 
decisions  may  no  longer  continue  to  provide  satisfying  outcomes  and  interest  inventory  results  can 
help  to  structure  the  process  of  the  individual’s  analysis  of  past  and  present  interests  and  priorities; 
(6)  reentering  the  iob  marked— interest  inventories  can  be  used  to  help  individuals,  especially 
women,  to  reenter  the  job  market;  (7)  facilitating  out-counseling-a  person  who  has  been  released 
from  his  or  her  job  can  use  this  time  to  reevaluate  occupational  interests  during  the  search  for  new 
employment;  (8)  preparing  for  retirement-examination  of  new  occupational  possibilities  and 
avocational  interest  assessment  via  interest  inventory  results  can  be  used  to  help  people  prepare  for 
the  change  from  full-time  work  to  leisure;  and  (9)  choosing  leisure  activities-Cairo  (1979) 
indicates  that  homogeneous  scales  of  interest  (e.g.,  Holland’s  Self  Directed  Search)  measure 
avocational  interests  as  well  as  occupational  interests  and  can  be  used  as  a  starting  point  for 
evaluating  leisure  interests. 

Personnel  Specialist  Applications.  This  includes:  (1)  selection-Reeves  and  Booth  (1979) 
found  in  a  study  using  the  Navy  Vocational  Interest  Inventory  (NVII)  that  adding  interest  measures 
to  an  aptitude  measure  significantly  increases  the  validity  for  predicting  occupation  effectiveness; 
and  (2)  placement— Flint  (1979)  described  the  use  of  the  Strong-Campbell  Interest  Inventory  (SCII) 
in  placement  and  promotion  of  police  officers  and  stressed  that  the  SCII  can  complement  other 
measures  such  as  personality  tests. 

Research  Applications.  This  includes:  (1)  personality  research-many  studies  have  been 
conducted  to  establish  construct  validity  of  vocational  interest  inventory  scales  by  determining  the 
relationship  between  personality  and  interests;  and  (2)  identifying  change  and  stability-interest 
results  can  be  used  to  detect  changes  over  time  in  interests  at  the  societal  level,  and  monitoring 
change  or  stability  of  interests  within  occupations  and  between  the  sexes. 

Hansen  (1981a)  concludes  that  with  increased  recognition  of  career  planning  as  a  means  to 
greater  self-satisfaction  in  life,  the  process  of  career  counseling  has  become  more  sophisticated  and 
used  to  help  a  client  integrate  his  or  her  past  history  with  current  and  future  goals. 

Methods  of  Scoring  and  Reporting  Interest  Results 

Two  types  of  scales,  heterogeneous  and  homogeneous,  are  used  to  measure  interests  (Hansen, 
1984).  Occupational  scales  developed  using  the  empirical  method  of  contrast  groups  are  called 
heterogeneous  because  items  selected  for  the  scales  have  low  correlations  with  one  another  (Berdie 
&  Campbell,  1968).  For  example,  the  207  Occupational  Scales  in  the  Strong-Campbell  Interest 
Inventory  are  heterogeneous  and  compare  the  interests  of  an  individual  to  the  likes,  dislikes,  and 
indifferences  of  men  and  women  already  in  each  of  the  occupations. 

In  contrast  to  heterogeneous  scales  homogeneous  scales  contain  items  that  have  a  high 
correlation  with  one  another.  Empirical  methods  of  cluster  analysis  and  factor  analysis  are  used  to 
identify  and  gather  together  related  items  for  homogeneous  scales;  these  scales  are  named  on  the 
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basis  of  the  scale’s  item  content  (Hansen,  1984).  An  example  of  homogeneous  scales  would  be  the 
Basic  Interest  Scales  developed  by  Campbell  in  the  Strong-Campbell  Interest  Inventory. 

A  second  type  of  homogeneous  scale  can  be  developed  by  the  rational  selection  of  items  and 
was  used  to  develop  Holland’s  six  General  Occupational  Themes,  which  appear  in  the  Self 
Directed  Search  (Hansen,  1984).  For  rational  scale  construction,  theory  is  used  to  determine  which 
items  in  an  item  pool  are  appropriate  for  each  scale  while  homogenous  scale  construction  using 
cluster  or  factor  analysis  is  a  strictly  statistical  and  atheoretical  procedure  (Hansen,  1984). 

According  to  Hansen  (1984),  heterogeneous  scales  have  the  best  predictive  validity  for 
determining  occupational  choice  but  homogeneous  scales  can  be  useful  in  exploring  a  client’s 
interests  as  they  relate  to  his  or  her  entire  life  style.  Together  the  heterogeneous  and  homogeneous 
scales  provide  a  system  of  checks  and  balances  that  allows  an  individual  to  identify  interests  in  a 
general  way  and  then  to  more  clearly  define  which  of  those  interests  are  occupationally  related 
using  the  heterogeneous  scales  (Campbell  &  Hansen,  1981). 

Debate  concerning  the  use  of  same-sex  (i.e.,  male  or  female)  norms,  combined-sex  (male  and 
female)  norms,  and  opposite-sex  (i.e.,  gender  opposite  that  of  the  test  subjects)  norms  for  reporting 
interest  assessment  results  has  historically  been  very  controversial  and  complex.  Until  the  1970s, 
most  interest  inventories  used  same-sex  norms  to  report  scores  (Seligman,  1980).  However, 
increasing  awareness  of  the  potential  for  sex  bias  and  stereotyping  seems  to  have  created  a  move 
away  from  the  use  of  same-sex  norms  and  toward  the  use  of  raw  scores  and  combined-sex  and 
opposite-sex  norms  (Seligman,  1980).  Both  Kuder  (1977)  and  Campbell  and  Hansen  (1981) 
concluded  that  failure  to  use  same-sex  norms  for  the  two  sexes  would  reduce  the  effectiveness  of 
counseling.  Hanson,  Noeth,  and  Prediger  (1977)  concluded  that  scores  based  on  same-sex  norms 
had  higher  predictive  validity  than  did  raw  scores  or  combined-  or  opposite-sex  norms  and 
suggested  a  less  traditional  range  of  occupational  options  than  did  other  methods  of  reporting 
interest  scores. 

Others  such  as  Tittle  (1974)  feel  that  reporting  same-  and  opposite-sex  norms,  such  as  in  the 
Strong-Campbell  Interest  Inventory,  reflect  cultural  stereotypes  and  are  another  way  of  restricting 
choices  for  females.  More  in-depth  reviews  concerning  the  issues  of  sex  bias  and  fairness  in  career 
interest  measurement  can  be  found  in  the  AM  EG  Commission  on  Sex  Bias  in  Measurement  (1977), 
Diamond  (1975),  and  Zytowski  and  Borgen  (1983). 

Some  have  argued  that  the  sex  differences  in  vocational  interests  that  early  researchers  found 
may  be  diminishing  in  response  to  growing  awareness  that  many  aspects  of  traditional  sex  roles  are 
arbitrary  (Diamond,  1975).  Hansen  (1981b)  has  studied  this  possibility  and  concluded  that  societal 
effects  do  not  seem  to  have  had  a  dramatic  effect  on  the  interests  of  women  from  the  1920s  through 
the  1980s  and  the  need  for  same-sex  norm  groups  still  exists. 

Studies  examining  client  satisfaction  with  different  kinds  of  interest  inventories  have  similar 
results  and  report  that  clients  have  been  equally  satisfied  with  the  SCII  and  Self  Directed  Search 
(O’Neil,  Price,  &  Tracey,  1979)  and  the  SCII  and  Vocational  Card  Sort  (Slaney,  1978).  Also,  few 
differences  were  found  in  the  effectiveness  of  different  methodologies  for  presenting  occupational 
information.  Group  approaches  to  interpretation  have  been  found  to  be  as  effective  as  individual 
approaches  by  several  researchers  (e.g.,  Smith  &  Evan,  1973;  Rubenstein,  1978).  Comparisons  of 
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various  technological  modes  for  presenting  inventory  results,  such  as  slides,  videotape, 
computer-assisted  and  written  interpretive  information,  also  suggest  equivalent  levels  of  client 
satisfaction  (e.g.,  Miller  &  Cochran,  1979;  Oliver,  1977). 

According  to  Hansen  (1984),  group,  video,  computer,  or  individual  presentations—rather  than 
written  interpretations-appear  to  provide  the  most  information  and  lead  the  client  to  participate 
most  fully  in  career  exploration.  But,  Hansen  states  that  clients  do  express  more  satisfaction  with 
the  interpretation  process  when  they  have  individual  interviews  with  a  counselor. 

Reliability  of  Selected  Interest  Measures 

Below  is  an  overview  of  the  test-retest  reliabilities  for  some  of  the  more  commonly  used  in¬ 


terest  inventories: 

1.  Strong-Campbell  Interest  Inventory  (Hansen,  1986): 

•  General  Occupational  Themes  .85 

•  Basic  Interest  Scales  .88 

•  Occupational  Scales  .89 

•  At  ademic  Comfort  Scale  .86 

•  Introversion-Extroversion  Scale  .90 

2.  Kuder  Occupational  Interest  Survey  (Zytowski  &  Kuder,  1986): 

•  Vocational  Interest  Estimates  .70-. 83 

3.  Self  Directed  Search  (Holland,  1985): 


(Note.  The  following  are  internal  consistency,  alpha,  measures.) 

•  Activity  Scales 

•  Competency  Scales 

•  Occupations  Scales 

•  Holland  Codes 

4.  Ohio  Vocational  Interest  Survey  (Domino,  1985): 

•  Occupational  Scales  .70s-. 80s 

As  pointed  out  in  Hansen  (1984),  interest  measure  reliability  is  often  confused  with  interest 
stability.  All  of  the  above  test-retest  reliabilities  were  computed  using  a  4-week  interval. 


.59-. 86 
.70-. 83 
.74-. 89 
.84-.92 
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Prediction  of  Job  Performance  and  Satisfaction 

Hansen  (1984)  reports  that  most  attempts  to  find  a  relationship  between  interests  and 
occupational  success  have  met  with  disappointment.  According  to  Tyler  (1965),  one  reason  for 
these  poor  results  is  that  most  studies  on  the  relationship  between  interests  and  success  are  limited 
by  the  use  of  only  one  indicator  of  job  success.  Another  major  problem  in  studying  the  relationship 
between  interests  and  success  is  determining  the  criterion  to  represent  success. 

Other  studies  that  have  used  more  innovative  designs,  such  as  using  ability  as  a  moderator 
(e.g.,  Clark,  1961)  found  some  interest  scores  more  predictive  of  success  at  some  ability  levels  than 
others.  Reeves  and  Booth  (1979)  found  that  adding  the  Hospital  Corpsman  (HM)  interest  scores 
from  the  Navy  Vocational  Interest  Inventory  to  verbal  and  arithmetic  aptitude  measures 
significantly  increased  the  validity  for  predicting  HM  effectiveness  from  .36  to  .41. 

Hansen  (1984)  suggests  that  combining  assessment  of  interests  with  assessment  of  abilities 
or  aptitudes  will  improve  prediction  of  achievement  or  success  in  academic  or  work  settings. 
Hansen  goes  on  to  state  that  as  a  rule,  persons  with  interest  in,  and  the  necessary  ability  for  an 
occupation  will  do  well  in  it;  persons  with  the  necessary  ability  but  not  the  interest  may  or  may  not 
do  well. 

Just  as  the  correlations  between  interest  and  job  performance  are  not  very  high,  correlations 
between  interest  results  and  ratings  of  job  satisfaction  are  also  low  (Hansen,  1984;  Gottffedson, 
1986).  According  to  Campbell  (1971),  the  primary  problem  is  the  lack  of  variance  in  satisfaction 
among  groups  with  homogeneous  interests  and  that  overall,  most  people  indicate  satisfaction  with 
their  work.  A  6  yeai  ;tudy  of  job  satisfaction  by  Weaver  (1980)  reported  that  between  1972  and 
1978,  88  percent  of  American  workers  said  they  were  somewhat  satisfied  or  very  satisfied  with 
their  job. 

In  contrast,  Gottfredson  (1986)  believes  the  problem  with  low  interest-satisfaction 
correlations  occurs  because  many  studies  combine  data  for  persons  working  in  diverse  occupations 
in  a  single  coefficient,  and  that  between -occupation  differences  in  satisfaction  are  great.  For 
example,  Kahn  (1972)  summarized  data  on  the  percentage  of  workers  in  different  occupations  who 
would  choose  similar  work  again.  The  percentages  ranged  from  93  percent  for  university 
professors  to  31  percent  for  textile  workers  and  16  percent  for  unskilled  workers. 

Gottfredson  cites  research  conducted  on  the  Vocational  Interest  Career  Examination 
(VOICE)  by  Alley,  Wilboum,  and  Berberich  (1976),  which  not  only  showed  that  job  satisfaction 
differed  greatly  among  occupational  groups  but  that  the  VOICE  Basic  Interest  Scales  showed 
incremental  validity  in  forecasting  job  satisfaction  within  occupational  groups.  Gottfredson  states 
that  these  results  are  consistent  with  other  research  in  which  Holland  scales  were  found  to  have 
usefully  high  correlations  with  job  satisfaction  in  homogeneous  occupational  groups  (Wiggins, 
1976).  Gottfredson  goes  on  to  state  that  one  useful  application  of  forecasting  job  satisfaction  within 
occupational  groups  may  be  in  matching  persons  with  occupations  for  which  satisfaction  is 
particularly  important  or  in  which  attrition  creates  special  difficulties  or  costs. 

In  conclusion,  Seligman  (1980)  asserts  that  interest  inventories,  for  the  most  part,  assess 
intrinsic  sources  of  satisfaction  on  the  likelihood  that  an  individual  will  enjoy  a  particular  type  of 
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work.  However,  Seligman  states  that  factors  regarding  extrinsic  satisfaction  such  as  working 
conditions,  salary,  and  life  style  inherent  in  a  particular  job  or  occupation  are  also  important  and 
manifest  themselves  in  a  person’s  expressed  interests.  Seligman  recommends  that  counselors 
should  be  sure  to  discuss  both  intrinsic  and  extrinsic  sources  of  satisfaction  with  their  clients  to 
determine  clients’  reactions  to  all  aspects  of  an  occupation. 

METHOD 

As  stated  in  the  overview,  the  primary  purpose  of  this  project  was  to  examine  the  feasibility 
of  developing  an  interest  assessment  instrument  for  use  primarily  in  the  vocational  counseling  of 
high  school  students  by:  (1)  assessing  counselor  preferences  on  the  format  and  results  reporting 
methods  the  instrument  should  exhibit;  and  (2)  assessing  the  acceptance  and  use  of  a  new  interest 
assessment  instrument  by  high  school  counselors. 

Subjects 

Subjects  for  the  study  were  high  school  guidance  counselors  from  a  local  school  district.  After 
obtaining  permission  from  the  Supervisor  of  Guidance  Services,  a  letter  was  sent  to  all  high  school 
counselors  in  the  district  (143)  inviting  them  to  participate  in  the  study.  A  $25  stipend  was  offered 
to  each  counselor  agreeing  to  participate  and  a  guarantee  that  all  individual  responses  would  be 
confidential.  A  total  of  12  counselors  responded  to  the  invitation;  eight  women  and  four  men  with 
one  black  male  and  one  black  female.  The  counselors  were  divided  into  two  discussion  groups  with 
six  counselors  in  each  group. 

Although  counselors  participating  in  this  study  may  not  be  representative  of  counselors-in- 
general,  one  inference  that  can  probably  be  made  is  that  they  are  most  likely  much  more 
sophisticated  than  counselors-in-general.  First,  the  school  district  for  which  these  counselors  work 
is  one  of  the  most  affluent  in  the  United  States.  Many  of  the  students  from  these  high  schools  go 
on  to  colleges  that  are  quite  competitive.  Second,  only  a  small,  self-selected  group  of  counselors 
agreed  to  participate  out  of  the  total  number  that  were  invited.  This  leads  one  to  infer  that  the 
counselors  agreeing  to  participate  are  probably  even  more  sophisticated  and  current  in  the  issues 
regarding  the  counseling  of  high  school  students  than  most  counselors  in  their  district  and  much 
more  than  counselors  in  general. 

Procedure 

All  counselors  agreeing  to  participate  were  sent  a  follow-up  letter  consisting  of  information 
on  how  to  get  to  the  meeting  site,  and  a  discussion  agenda.  The  follow-up  letter  also  asked  the 
counselors  to  imagine  that  if  they  could  develop  a  new  interest  assessment  instrument  for  high 
school  students,  what  design  features  would  they  want  to  have  included.  Counselors  were  asked  to 
write  down  their  top  five  preferences  in  rank  order  and  bring  them  to  the  meeting  where  they  would 
be  collected  by  the  moderator  prior  to  starting  the  discussion  group.  The  reasons  for  asking 
counselors  to  think  about  and  write  down  their  preferences  before  coming  to  the  discussion  groups 
were  to:  (1)  stimulate  counselor  thinking  regarding  issues  surrounding  the  measurement  of 
interests  and  about  their  preferences  for  a  new  interest  assessment  instrument  prior  to  convening 
the  discussion  group;  and  (2)  allow  counselors  time  to  think  about  their  preferences  without  being 
influenced  by  the  opinions  of  their  peers. 
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This  brief  survey  also  gave  the  moderator  a  quick  preview  of  counselor  preferences  that 
should  be  addressed  first  by  the  group  and  which  may  not  otherwise  have  been  considered  by  the 
moderator. 

A  small  group,  structured  interviewing  approach  (i.e.,  moderated  discussion  groups)  was  used 
as  the  means  for  data  collection.  During  the  discussion  groups,  other  issues  that  were  of  importance 
to  counselor  acceptance  of  a  new  interest  assessment  instrument  were  also  addressed. 

All  discussion  groups  were  moderated  by  a  professional  thoroughly  familiar  with  issues 
regarding  the  use  of  interest  assessment  instruments  in  high  schools.  The  length  of  each  discussion 
group  was  about  1.5  hours.  Discussions  were  tape  recorded  with  the  prior  consent  of  all 
participants.  Most  of  the  counselors  were  familiar  with  one  another,  generally  very  personable,  and 
quite  eager  to  talk  about  interest  assessment  in  high  schools.  The  moderator’s  role  was  primarily 
to  provide  structure  to  the  discussions  by:  (1)  ensuring  that  all  participants  had  a  chance  to  voice 
their  opinions  and  that  no  one  participant  dominated  the  discussions;  (2)  encouraging  participants 
who  were  not  speaking  much  to  participate  by  directly  soliciting  their  opinions  on  different  topics; 
(3)  ensuring  that  the  group  did  not  dwell  too  long  on  any  one  topic;  and  (4)  ensuring  that  all  topics 
were  addressed  from  the  topic  guide  and  the  survey  of  counselor  preferences. 

The  moderator  also  had  to  be  aware  of  new  issues  of  importance  that  arose  during  the 
discussions  and  which  therefore  needed  to  be  addressed  by  the  group. 

Before  conducting  the  discussion  groups,  the  moderator  prepared  a  topic  guide.  Below  is  a 
list  of  the  points  that  were  included  in  the  topic  guide: 

1.  Should  the  instrument  items  be  occupational  titles,  activities  associated  with  occupa¬ 
tions,  or  a  combination  of  both? 

2.  Should  the  new  instrument  measure  expressed  interests,  inventoried  interests,  or  both? 

3.  In  what  way  should  the  instrument  report  results  (e.g.,  occupational  scales,  occupation¬ 
al  themes,  etc.)? 

4.  Should  the  instrument  results  be  reported  in  terms  of  same-sex,  opposite-sex,  or  com¬ 
bined-sex  scores? 

5.  How  would  you  prefer  the  instruments  be  scored  (i.e.,  self-  or  computer-scored)? 

6.  By  whom  should  the  results  of  the  instrument  be  interpreted  (i.e.,  self,  counselor, 
group)? 

7.  What  is  the  maximum  length  of  time  it  should  take  to  complete  the  interest  assessment 
instrument? 

8.  How  much  should  the  instrument  cost  to  administer? 


9.  How  important  is  the  scientific  soundness  of  the  instrument? 

10.  How  should  the  new  instrument  be  formatted? 

Other  topics  of  importance  to  the  counselors  were  also  raised  and  addressed  during  the 
discussions. 


RESULTS 

Below  are  the  discussion  group  results  organized  according  to  the  talking  points  in  the  above 
topic  guide.  Also  included  are  the  results  from  an  interview  with  the  Superintendent  of  Guidance 
(SOG)  of  the  school  district.  Additional  topics  of  importance  to  the  counselors  and  SOG  but  not 
included  in  the  topic  guide  are  also  described. 

Item  Type 

Counselors  were  asked  if  they  preferred  the  new  instrument  to  contain  items  that  had  the 
student  respond  to  occupational  titles,  activities  associated  with  occupations,  or  a  combination  of 
both.  The  counselors  were  almost  unanimous  in  their  preference  for  the  instrument  items  to  be  only 
activities  associated  with  occupations.  They  did  not  like  the  idea  of  using  occupational  titles 
because  of  their  susceptibility  to  stereotypes,  etc.  The  counselors  were  particularly  concerned  that 
the  salaries  associated  with  certain  occupations  would  have  a  very  strong  influence  on  how 
students  responded  to  occupational  titles.  The  counselors  have  reason  for  concern  since 
occupational  titles  have  some  established  stereotypical  reliability  (Baker,  1983). 

Expressed  vs.  Inventoried  Interests 

Overwhelmingly,  counselors  did  not  like  the  idea  of  the  new  instrument  assessing  expressed 
interests  only.  They  felt  expressed  interests  were  too  susceptible  to  factors  such  as  occupational 
misconceptions  and  associated  salaries  and  preferred  an  instrument  with  items  describing 
activities.  When  the  moderator  probed  further  and  asked  how  they  would  respond  to  an  instrument 
that  included  both  expressed  and  inventoried  interests  so  that  inventoried  and  expressed  results 
could  be  compared,  the  counselors  were  much  more  responsive.  They  felt  this  would  be  a  good  idea 
because  the  expressed  interests  include  aspects  of  desired  life-style,  which  are  not  included  in  the 
inventoried  results. 

Seligman  (1980)  would  agree  with  this  reasoning  and  states  that  most  interest  inventories 
measure  primarily  intrinsic  sources  of  satisfaction  without  considering  outside  or  extrinsic  sources 
of  satisfaction  such  as  working  conditions,  salary,  and  life-style.  Hansen  (1984)  also  states  that  the 
preferred  mode  of  career  counseling  today  is  to  consider  both  expressed  and  inventoried  interests. 
In  contrast,  both  Baker  (1983)  and  Slaney  and  Slaney  (1986)  cite  recent  and  considerable  evidence 
supporting  the  predictive  validity  of  expressed  interest  measures. 
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Results  Reporting  Format 


In  general,  counselors  favored  the  use  of  basic  interest  scales  or  occupational  themes  (i.e., 
homogeneous  scales)  over  the  use  of  occupational  scales  (i.e.,  heterogeneous  scales).  Counselors 
were  almost  unanimous  in  their  belief  that  students  in  high  school  needed  to  explore  their  interests 
in  terms  of  how  they  relate  to  their  entire  life  style  rather  than  just  how  their  interest  results 
compare  with  those  of  people  in  many  different  occupations.  As  discussed  in  Hansen  (1984),  this 
is  the  greatest  benefit  of  homogeneous  scales  over  heterogeneous  scales.  The  primary  criticism 
counselors  had  against  occupational  scales  was  that  they  tended  to  overload  the  student  with 
information  and  that  for  this  time  in  a  student’s  life,  exploration  of  interests  was  more  important 
than  occupational  prediction. 

Also,  counselors  felt  some  method  for  linking  interest  results  to  areas  of  study  in  college 
would  be  very  beneficial.  They  explained  that  many  of  their  students  went  on  to  college  and  that 
they  were  much  more  concerned  about  selecting  a  college  major  than  a  specific  occupation.  They 
asserted  that  such  a  linkage  would  enable  the  interest  measure  results  to  be  used  for  counseling  both 
college-bound  and  non  college-bound  students. 

Sex  Bias 

When  asked  if  the  new  instrument  results  should  be  reported  in  terms  of  same-sex,  opposite- 
sex,  or  combined-sex  scales,  the  counselor  responses  were  very  mixed.  First,  most  of  the 
counselors  were  not  familiar  with  the  pros  and  cons  regarding  the  use  of  such  scales  and  did  not 
consider  scale  type  a  major  factor  in  their  decision  to  use  a  new  instrument.  This  is  not  surprising 
given  the  complexity  and  controversial  nature  of  this  topic  among  researchers.  Many  of  the 
counselors  felt  that  same  sex  scales  were  sexist  because  they  attempted  to  measure  the  interest  of 
men  and  women  differently.  Some  counselors  felt  there  was  no  need  for  same  sex  scales  because 
women  today  believe  they  can  do  anything  (although  the  research  of  Hansen  (1984)  reached  a 
different  conclusion).  In  general,  counselors  felt  only  one  combined-sex  scale  was  needed. 

If  one  assumes  that  these  counselors  are  probably  much  more  sophisticated  than  counselors- 
in-general,  then  most  counselors  in  the  U.S.  probably  do  not  understand  the  issues  regarding  sex 
bias  in  reporting  interest  assessment  results.  One  conclusion  that  can  be  made  is  that  no  matter  what 
scale  might  be  used  for  the  new  interest  measure,  a  clear  and  thorough  explanation  in  the 
instrument  manual  will  be  needed  to  explain  how  the  scales  should  be  interpreted  by  the  counselor. 

Reviewing  the  course  of  both  discussion  groups,  it  was  noted  that  the  use  of  same-sex  scales 
certainly  did  not  seem  to  decrease  counselor  enthusiasm  for  certain  instruments  and  that  possibly 
this  is  really  not  that  important  of  an  issue  for  counselors.  For  example,  during  both  discussion 
groups  counselors  expressed  praise  for  the  ASVAB  and  Strong-Campbell  Interest  Inventory,  both 
of  which  use  separate-sex  scales  to  report  aptitude  and  interest. 

Scoring 

All  of  the  counselors  preferred  that  the  instrument  be  scored  by  the  student  and  not  have  to  be 
sent  out  for  computer  scoring.  They  preferred  two  results  sheets  be  included,  one  for  both  the 
student  and  counselor.  Many  of  the  counselors  stressed  the  value  of  immediate  feedback,  which 
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could  be  obtained  through  self-scoring.  In  theory  all  of  the  counselors  felt  that  in  the  future,  a 
computer  administered  and  scored  instrument  would  be  desirable  but  that  currently  there  were  not 
enough  computer  terminals  available  to  them  to  make  this  feasible. 

Interpretation  of  Results 

Along  with  being  self-scoring,  counselors  stated  the  new  instrument  should  also  be  self¬ 
interpretable.  The  counselors  felt  the  results  sheet  should  have  very  clear  documentation 
accompanying  it  to  help  the  students  and  their  parents  interpret  the  results.  Also,  the  documentation 
should  encourage  students  to  discuss  their  results  with  a  school  guidance  counselor. 

Time 

The  majority  of  counselors  felt  that  the  instrument  should  be  such  that  even  the  slowest  stu¬ 
dent  would  be  able  to  complete  it  in  about  30  minutes.  Counselors  explained  that  the  30  minute 
completion  time  was  necessary  because  often  they  had  to  give  the  instrument  within  a  45  minute 
class  period.  Although  there  was  some  discussion  that  the  instrument  could  be  completed  at  home 
and  did  not  need  to  be  restricted  to  30  minutes,  many  of  the  counselors  still  preferred  that  the  in¬ 
strument  be  no  longer  than  30  minutes.  Time  was  also  a  factor  considered  by  the  SOG  when  re¬ 
viewing  any  instrument  because  testing  time  usually  cuts  into  class  time. 

Cost 


A  very  big  factor  for  both  the  counselors  and  the  SOG  was  that  cost  alone  would  not 
determine  their  decision  to  use  a  new  interest  measure  and  that  both  cost  and  value  would  have  to 
be  considered.  Several  counselors  expressed  their  dislike  for  instruments  where  they  had  to  pay  for 
both  the  instrument  and  for  having  it  scored.  In  order  to  reduce  the  cost  of  a  new  interest  measure, 
many  of  the  counselors  recommended  having  reusable  test  booklets  with  separate  answer  sheets 
that  could  be  kept  by  the  students.  Counselors  also  noted  that  the  less  expensive  a  test’s 
administration  costs,  the  more  students  that  could  be  tested. 

The  level  of  importance  that  the  counselors  and  SOG  gave  to  cost  is  notable  considering  that 
they  work  in  one  of  the  more  affluent  school  districts  in  the  U.S.  One  can  probably  infer  that  if  cost 
is  of  concern  to  this  group  of  counselors,  then  it  would  be  of  concern  to  almost  all  personnel  in  the 
U.S.  who  are  in  charge  of  selecting  assessment  instruments  for  school  use.  The  counselor  and 
SOG’s  concern  with  instrument  cost  is  also  in  agreement  with  the  results  a  survey  of  schools 
conducted  by  Engen,  Lamb,  and  Prediger  (1982),  which  found  that  72  percent  of  the  schools 
surveyed  would  administer  more  tests  if  they  had  more  time  and  money. 

Scientific  Soundness 

Although  the  scientific  soundness  of  a  new  interest  instrument  was  not  of  great  importance  to 
the  counselors,  it  was  very  important  to  the  SOG.  The  SOG  stated  that  all  instruments  that  are  used 
in  the  district  must  be  first  approved  by  a  review  committee.  Some  of  the  criteria  the  committee 
considers  when  reviewing  different  instruments  are  factors  such  as  the  instrument’s  validity, 
reliability,  norm  group  used,  etc.  The  counselors  did  state  they  would  not  want  to  use  an  instrument 
that  reported  biased  results. 
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Instrument  Format 


Neither  the  counselors  nor  the  SOG  had  any  strong  preference  as  to  how  the  instrument 
should  be  formatted.  In  general,  they  thought  the  instrument  should  be  organized  to  be  very 
readable  and  that  the  type  should  not  be  too  small  (a  problem  several  of  the  counselors  noted  in 
some  of  the  currently  available  instruments).  The  counselors  also  felt  that  the  layout  should  be 
attractive  to  students  but  not  include  so  many  "bells  and  whisdes"  that  it  would  significandy 
increase  the  instrument’s  price.  They  also  felt  the  instrument  should  be  designed  so  it  does  not 
appear  that  completing  it  will  be  an  overwhelming  task  for  the  student. 

Integration  of  Interest  Results  With  Aptitude  and  Occupational  Information 

Although  this  issue  was  not  on  the  moderator’s  topic  guide,  it  was  an  extremely  important 
topic  for  the  counselors.  In  both  the  counselors’  survey  results  and  during  discussion  groups,  all  of 
the  counselors  said  there  were  very  few  interest  instruments  available  today  that  linked  interests  to 
either  aptitude  or  occupational  information  and  that  having  this  feature  in  a  new  interest  instrument 
would  be  a  major  selling  point.  They  felt  current  instruments  that  attempt  to  make  these  links  either 
do  not  do  a  very  good  job  or  are  extremely  difficult  and  complex  to  use. 

Counselors  explained  that  the  link  with  aptitudes  was  important  because  a  student’s  interest 
results  may  highlight  occupations  for  which  the  counselor  knows  the  student  does  not  have  the 
required  level  of  aptitude.  Having  a  link  with  aptitude  would  direct  students  to  occupations  they 
are  not  only  interested  in  but  also  have  the  required  level  of  aptitude.  In  both  discussion  groups, 
counselors  felt  the  ASVAB  was  one  of  the  better  tests  of  aptitude  being  used  in  schools  today  and 
found  its  results  reporting  format  to  be  very  clear  and  understandable.  When  asked  about  their 
acceptance  of  an  interest  measure  linked  with  ASVAB  results,  all  of  the  counselors  stated  this 
would  be  a  great  idea.  (Note.  Counselors  were  not  told  the  name  of  the  client  until  the  end  of  each 
discussion  group  and  any  remarks  by  counselors  regarding  the  DoD  Student  Testing  Program  and 
ASVAB  were  spontaneous.) 

Counselors  believed  a  link  with  occupational  resources  was  important  because  more  is 
needed  than  just  highlighting  for  a  student  a  group  of  occupations  he  or  she  might  be  interested  in. 
Counselors  felt  the  documentation  for  helping  students  to  interpret  their  interest  results  should  also 
encourage  students  to  learn  more  about  the  occupations  they  are  interested  in  by  directing  them  to 
occupational  resources  such  as  the  Occupational  Outlook  Handbook  (OOH).  Counselors  also  felt 
the  occupational  titles  used  in  the  interest  results  should  be  compatible  with  those  used  in  sources 
of  occupational  information  such  as  the  OOH. 

Reading  Level 

Reading  level  also  was  not  on  the  moderator’s  topic  guide  but  was  of  some  importance  to 
counselors  and  the  SOG.  They  explained  that  all  directions,  documentation,  and  items  should  be 
very  readable  and  use  a  simple  vocabulary.  Counselors  felt  the  instrument’s  reading  level  should 
be  documented  in  the  counselor’s  manual. 
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CONCLUSIONS 


The  design  features  this  group  of  counselors  would  prefer  in  a  new  interest  assessment 
instrument  are  in  rank  order: 

1 .  Results  linked  with  a  measure  of  aptitude  and  with  sources  of  occupational  information. 

2.  Inexpensive  and  effective. 

3.  Unbiased  results  for  women  and  minority  groups. 

4.  Self  scoring  and  interpretable  by  student,  with  materials  provided  to  counselors  for 
more  extensive  interpretation. 

5.  Administration  time  should  be  about  30  minutes. 

6.  Results  should  be  in  terms  of  occupational  themes  such  as  Basic  Interest  Scales  and/or 
Holland  codes. 

7.  Items  should  be  in  terms  of  activities  associated  with  occupations  vs.  occupational  ti¬ 
tles;  combined  assessment  of  expressed  and  inventoried  interests  was  also  thought  to  be  very  use¬ 
ful. 


8.  All  documentation,  items  and  results  should  be  written  at  an  appropriate  reading  level. 

9.  Results  should  be  reported  in  terms  of  combined-sex  scores. 

10.  Instrument  should  be  attractively  formatted  and  easy  to  read. 

The  above  ranking  of  counselor  preferences  was  done  subjectively  by  the  moderator  and  does 
not  constitute  an  empirical  ranking. 

Computerization  was  often  discussed  by  the  counselors.  They  stated  that  computers  could 
help  them  greatly  to  integrate  and  improve  the  delivery  of  career  guidance  services  to  students 
although  they  currently  did  not  have  enough  hardware  available  to  take  advantage  of  existing 
career  guidance  software.  Counselors  felt  this  situation  will  be  resolved  as  hardware  becomes 
cheaper  and  more  available.  Several  of  the  counselors  had  seen  demonstrations  of  videodisk  career 
guidance  software  and  were  very  impressed.  Borgen  (1986)  and  Cairo  (1983)  also  believed  that 
computers  and  videodisk  technology  will  be  the  wave  of  the  future  for  career  counseling.  Hence, 
the  new  interest  assessment  instrument  should  be  designed  so  that  it  can  be  easily  administered  via 
paper-and-pencil  and/or  computer. 
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